A Framework for Cut-Off Incremental Recompilation and Inter-Module Optimization

In this paper we present a cut-off incremental recompilation framework that supports inter-module optimization. The framework allows arbitrary compile time information to propagate across program unit boundaries, in such a way that it can be determined if compilation assumptions have changed since the program unit was last compiled. The abstract presentation of the framework makes explicit the assumptions of the approach and specifies exactly the set of operations necessary for each of the translation phases in a compiler (from parsing to abstract syntax, through a series of intermediate languages, and eventually down to machine code). The correctness results shown are non-trivial due to the flexibility of the framework, which allows even open terms (objects containing free occurrences of names) to propagate across program unit boundaries at compile time. The framework is based on a language for programming in the very large, which allows for expressing precise program unit dependencies. Moreover, framework is batch-based in the sense that only one source file is compiled for each invocation of the compiler. The scheme is applied in the MLKit Standard ML compiler and experiments show that the approach is feasible and that it scales to very large programs and day-to-day program development for programs consisting of more than 250.000 lines of Standard ML.